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Abstract: We propose a new Bayesian Network model that segments the region of interest from the image and categorize the 
segmented image. A Bayesian Network is constructed from an over segmentation to model the statistical dependencies 
among edge segments, vertices, corners and blobs. The proposed interactive image segmentation involves active user 
intervention for interactive image segmentation while existing interactive segmentation approaches are often passively 
depend on the user to provide exact intervention. Image categorization is done based on the Spatial Markov Kernel (SMK) 
that categorizes the segmented image. 

Index Terms: Bayesian Network, Image Segmentation, Interactive image segmentation. Image Categorization, 
Spatial Markov Kernel algorithm 

I. Introduction 

For some applications, such as image recognition or compression, we cannot process the whole image directly for 
the reason that it is inefficient and unpractical. Image segmentation (IS) and categorization is a difficult problem in computer 
vision. IS ai ms at partitioning an image into constituent regions of interest and identify category of the segmented image. 
Data-driven approaches sometimes fail to produce accurate segmentation when the image contains shadow, occlusion, 
cluttering, low contrast area, or noise. The existing systems including the clustering method, region growing, active contours, 
normalized cuts [1], graph-cut-based approaches [2], Markov random fields (MRFs), etc. adopts Data-Driven approach and 
thus the Bayesian network model is adopted for accurate image segmentation. 

The term image categorization refers to the labeling of images into one of a number of predefined categories. 
Although this is usually not a very difficult task for humans, it has proved to be an extremely difficult problem for machines 
(or computer programs). The existing works of image categorization are done with regions and colors they are low level 
categorization of images which doesn't offer a much accurate result when the background and the foreground are in the same 
color or dimension so to overcome this Spatial Markov Kernel is used. 



II. Overview of the Approach 

The aim of the proposed system is to segment the region of interest and categorize the segmented image based on 
the available dataset. The overall approach of the proposed system is clearly portrait in the flow chart diagram "Fig 1". The 
steps involved in this system are 

1. Input the image to be segmented with the outline image. Outline image is the user intervention image that details the 
region to be segmented from the whole image 

2. Construct the over segmented image correlating the statistical dependencies among edge segments, vertices and corners 

3. Construct a Bayesian network with region, edge, vertex measurements as nodes 

4. Segmentation is carried out by matching the pixels 

5. If the user is satisfied with segmented output it is classified further else re-segmentation is carried out 

6. The segmented output with its category is output to the user. 
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Fig 1 flow chart of proposed system 
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The over segmentation of the image is obtained from standard image segmentation approach like watershed 
segmentation [3], normalized cut, and anisotropic diffusion. In our experiment we have used watershed segmentation to 
obtain the over segmentation image. 

III. Multiclass Image Segmentation 

Multiclass image segmentation can be done in two different ways 

1 . We can design a series of BN models for segmenting each class versus the background. Applying these models will 
generate a multiclass segmentation. 

2. We can extend the binary nodes (especially the region nodes) in the BN to discrete nodes with multiple states, allowing 
distinguishing multiple object classes. 

The second method is followed in our experiments i.e. by extending the region node to discrete nodes with multiple states. 

IV. Image Categorization Algorithm 

1. Given a multiclass image categorization problem, the strategy of training an Support Vector Machine[4] (SVM) 
classifier is one against the rest for our SMK2 and one against one for our SMK1 

2. we need to compute our SMK2 C times to solve an image categorization problem with C categories based on the one- 
against-the-rest strategy, whereas our SMK1 can be commonly adopted for all the binary classification sub problems 

3. An image with X x Y blocks can be denoted as a sequence Q = ql,lql,2, . . . ,ql,Y q2,lq2,2, . . . , q2,Y , . . . , qX,Y , 
where qx,y C S (1 < x < X, 1 < y < Y ) is the visual keyword automatically assigned to block (x, y) in the image. 

4. Let Q x,y denote the sequence of states from block (1, 1) to block (x, y) in a row wise raster scan on the regular grid 

SMK1: 

The basic idea of defining a kernel is to map the 2-D sequence Q of an image into a high-dimensional feature space: 
Q— »0(Q). If an SMC model X (Q) is estimated via MLE for each image (or each sequence) Q, the feature mapping O can be 
given as 

<D(Q) = ^ Q) ={7i (Q) ,H (Q) ,V (Q) } (1) 

That is, each sequence Q is now represented by the model parameters of an SMC. 

We focus on capturing spatial dependence between states (visual keywords), we only consider the horizontal and 
vertical transition matrices in ®(Q). Moreover, to make the computation more efficient, the feature mapping O is then 
defined as 

O(Q) = { (h ;j (Q) + VlJ (Q) )/2 } i<i,j< M (2) 



The feature mapping O has been defined; our SMK1 function in the feature space (determined by O) can be given as 
K(Q,~Q) = K 0rig (O(Q),OCQ)) (3) 



Where Q and ~Q are two sequences and K olig can be any kind of an original kernel. In our experiments, the Gaussian 
kernel is used as Ko rig 



SMK 2: 

Let XO = [no, HO = {h° 1J } MxM , VO = {v° 1j } M xm} be the SMC model estimated for a category. The feature mapping O 
for a sequence Q can be defined based on the following Fisher score: 



<D(Q) = 5 log P(Q\X) 



(4) 



Where X = {ti,H,V} denotes an arbitrary SMC model. Since we focus on capturing the spatial dependence between 
states (visual keywords), only the horizontal and vertical transition 

matrices (i.e., H and V ) are used to define O(Q) in the following. To make sure that the two transition matrices H 
={hij}MxM and V = { } Mx m satisfy the probability constraint conditions (i.e., X M j=i n ij = 1 £ M j=i vr j = 1)> we ta ke into 
account the following substitutions: 



=e mj /^ =; e pij - 



(5) 
(6) 
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The feature mapping used by SMK1, we finally take into account the following O for the SMK2 function 

4>(0={( y (e) + v p« (G) )/2 )msn (7) 

V. Experimental Setup 

Our proposed model is implemented by MATLAB version 7 and tested. An example with the jaguar image is given 
below. The outline image is given as input along the jaguar image for segmentation fig (3) and fig (2). 




The user intervention i.e. the region to segment is given through the outline image. The region above the blue 
colored lines are ignored and the pixels that matches with the blue line is considered and segmentation is carried out. Initially 
the oversegmented image is generated. Outline image over oversegmented image and oversegmented image is shown in the 
fig (4) and fig (5) respectively 





Fig 4: outline over oversegmented image Fig 5: oversegmented image 

The fig(6) and fig(7) shows the segmented region and its background region which is the required output of our system. 




Fig 6: Segmented image 



Fig 7: Background image 
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The segmented image is then categorized based on the information available in the database. The output after the 
segmented image categorization is given in fig (8) 




The image is categorized as 

Jaguar 

Rock 

Grass 



Fig 8: Output image 



VI. Conclusion 

In this paper, we first propose a Multiclass image segmentation with image categorization based on Bayesian 
Network. The Bayesian Network systematically encodes the relationships among regions, edge segments, vertices, angles, 
and their measurements. The advantages of our proposed system is 

1. Implementation of multiclass image segmentation and categorization. 

2. It provides a systematic way to model the image segmentation problem in a probabilistic manner. 

3. It is very convenient to incorporate new knowledge into the model due to the expressive and inference power of 
Bayesian Networks. 

4. The user's intervention can be easily incorporated into the Bayesian Network as new evidence in an incremental way 
unlike other IS methods 

The accuracy obtained by our model is 93.5%. Finally, although our model focuses on Multiclass Image 
segmenation and categorization the model is designed for dataset other than medical images which will be studied in our 
future along with the connectivity constraints. 
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